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ABOUT THIS CASE STUDY 


Achieving the Dream (ATD) is one of 12 higher education and digital 
learning organizations that make up the Every Learner Everywhere (Every 
Learnen Network, whose mission is to help higher education institutions 
improve and ensure more equitable student outcomes through advances 
in digital learning, particularly among poverty-impacted, racially 
minoritized, and first-generation students. Every Learner partners are 
addressing high failure rates in foundational courses through the provision 
of scalable, high-quality support to colleges and universities seeking to 
implement adaptive courseware on their campuses. As part of its ongoing 
effort to help community colleges develop effective teaching and learning 
practices, ATD is working with seven community colleges in Florida, Ohio, 
and Texas on this initiative, providing coaching and direct support to the 
colleges, fostering collaboration within and among the participating 
institutions, and serving as a liaison to the Every Learner network. 


The following case study is part of a series of studies conducted by ATD 
examining how adaptive courseware is implemented at those institutions 
as well as how courseware is used in particular disciplines to better serve 
students. Case studies are based on a series of interviews with college 
leaders, faculty, instructional designers, developers, technology specialists 
and students who were enrolled in classes using the Courseware. 
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OVERVIEW 


Mathematics is a required component of many 
community college degree pathways, yet large numbers 
of students come to college needing additional support 
and struggle to complete gateway math courses— 
courses in which low-income and minority students are 
often overrepresented. All seven community colleges 
participating in the Every Learner Network implemented 
adaptive courseware fo help students succeed in 
gateway math courses. Among their findings: 


e Adaptive courseware provided 


needed corequisite and/or basic 
skills support, particularly at 
institutions which had eliminated 
developmental education 
courses as part of state-level 

or institutional reforms; some 
institutions also deployed the 
technology in higher-level math 
courses based on the experiences 
of students in introductory courses. 


Faculty leadership in 

identifying and deploying 
adaptive courseware was 
critical, particularly given the 
importance of ensuring that 

the courseware aligned with 
curricula and classroom lectures, 
activities, and assessments. 


Faculty and students stressed 
the importance of introducing 
students to courseware, 
regularly monitoring the student- 
performance data it generates 
to target class content and 
individual interventions, and 
aligning courseware with out-of- 


class supports such as tutoring or 
corequisite labs. 


e Scaling the use of courseware 


varied from institution to 
institution, but efforts often 
focused on providing a consistent 
integrated experience across 
course sections taught by full-time 
and adjunct faculty. 


BSUPPORTING INSTITUTIONAL REFORM 


The Every Learner initiative supports broader efforts to 
foster student learning with evidence-based practices, 
including supporting students in gateway courses in 
which low-income and minority students are often 
overrepresented. “Out of the gate, we were having more 
meaningful conversations about what students need,” 
says Sarah Wyatt, associate professor of mathematics 
at Indian River State College. It also reflects the contexts 
in which broader institutional reform is taking place 

at community colleges throughout the ATD Network, 
including building a culture of excellence in teaching 
and learning and leveraging data and technology 

To support student success and equitable student 
outcomes. To learn more, see p. 8. 
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COURSE AND COURSEWARE 


INSTITUTION ore} U SS) 3) COURSEWARE 


Broward College 


Fort Lauderdale, FL Developmental Mathematics ALEKS (McGraw-Hill) 
38,976 students 
Indian River State College Intermediate Algebra 
; MyLab Math (Pearson) 
Fort Pierce, Florida College Algebra é 
ALEKS (McGraw-Hill) 
16,686 students Quantitative Reasoning 


Miami Dade College 
Miami, FL College Algebra Integrated Review (Pearson) 
51,679 students 


Cuyahoga Community College 
Basic Arithmetic /Pre-Algebra Knewton Alta (Wiley) 


Cleveland, OH 

Beginning Algebra | MyLab Math (Pearson) 
23,655 students 
Lorain County Community 
College 
EV icrer Statistics WileyPlus Adaptive Practice 
10,206 students 

College HSE Math 
Amarillo College STEM College Algebra 
Amarillo, TX College Algebra MyLab Math (Pearson) 
9,739 students Business Math 


Contemporary Math 


Houston Community College College Algebra 
Houston, TX Math for Business and Social Knewton Alta (Wiley) 
56,151 students Sciences 


For more detail on the pilot efforts at each college, see case studies for each institution. 
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INTRODUCTION 


Geolmary Suazo admits to being anxious before taking 
her first math class at Broward College in Florida. 


“| was nervous because it was 
my first math class in another 
language,” says Suazo, who 
immigrated from Nicaragua the 
year before enrolling at Broward. 
“| thought it was going to be very 
hard, and | remembered that | 
had a classmate tell me that it 
was the third time she was taking 
the course.” 


Suazo isn’t alone. Gateway 
courses in math are mandatory 
for many community college 
students, regardless of intended 
major or program, and few begin 
their postsecondary studies 
prepared to excel. Efforts to reform 
developmental education by 
placing studenis into college-level 
courses regardless of ability level 
in some states have placed even 
greater emphasis on meeting 
students where they are and 
assisting them through these crucial 
introductory courses. 


A cross-section of mathematics 
faculty across community 
colleges participating in the Every 
Learner Network grant suggests 
that adaptive courseware has 
benefits across the full soectrum 
of ability levels—from self-guided 
remediation and support for 
struggling students to helping 
reinforce skills for Those who are 
ready for college-level content. 


A common strength is the 
technology's ability fo provide 
guided practice outside of the 
classroom setting. 


“They can keep working at it until 
they get it. They can get help. They 
can follow up,” says Kati Dobeck, 
a mathematics professor at Lorain 
County Community College 
(LCCC) in Ohio. “They don’t have 
to stop if they’re not happy with a 
score they earned.” 
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CREATING ALIGNED SUPPORTS 


Supporting students in gateway courses is particularly 
important at institutions in states where legislation 

or policy changes have prompted changes in 
developmental education. At Miami Dade College, for 
example, adaptive software was intended to support skill 
building in students following legislation passed in 2013 
that effectively eliminated develoomental education 
math courses. “It was intended to take care of students 
on the back end so they could catch up and (be) all 
the same in the classroom,” says Nicholas Schur, math 
department chair at Miami Dade College’s (MDC) 


Kendall Campus. 
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—Jennine Wilson, Indian River State College student 


Beyond support for skill building, 
faculty found that adaptive 
courseware also benefitted 
students unfamiliar with the 
workload and expectations 

of college work. At LCCC, for 
example, Dobeck uses adaptive 
courseware for college-level 
statistics courses—the college's 
largest enrollment math course— 
but not for corequisite support, 
which is provided by a paired 
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developmental course focused 

on small-group instruction. 
Instead, adaptive courseware was 
intended to help students build the 
kind of study skills required to be 
successful in college-level math 
courses. “So many students can 
get by (in high school) without 
doing any studying,” she says. 

“If you don’t get it yet, you’re not 
proficient yet. There’s more work 
you can do. It’s interesting how 
many ways it can be used.” 


However, math faculty stressed that 
adaptive software is not a set-and- 
forget add-on to curriculum. They 
emphasized the importance of 
aligning courseware to curriculum 
and the content of in-class lectures. 
“We ensure the materials we 
choose are covered within our 
outlines,” says Jasmine Vazquez, an 
assistant professor of mathematics 
at Broward College. 


Faculty also stressed the importance 
of introducing students to courseware 
and explaining the rationale behind 
it at the beginning of Courses, as 

well as regularly monitoring student 
performance and intervening 

as needed. A common strategy 
involved adjusting grading thresholds 
and other components to ensure that 
students didn’t become frustrated 

by a lack of progress within the 
courseware, which typically requires 
set pass levels before students can 
move on to graded assessments or 
the next unit of the course. 


“It was challenging to develop how 
we were going to use it,” says Christine 
McDonald, a master instructor in 
mathematics at Indian River State 
College (IRSC) in Florida. “We fumbled 
around for a couple of semesters 
figuring out what to use that wouldn’t 
overburden students but be enough.” 
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everything how it can be overwhelming.39 
—Brandon Rios, Miami Dade College student 


Some institutions also made 
efforts to align out-of-class 
supports, including faculty and 
peer tutoring, to support students 
by working directly within the 
courseware. MDC, for example, is 
adding a lab for college algebra 
students built around the same 
adaptive courseware platform. 
“Whatever the courseware is 
telling them to work on, they can 
get help in the lab,” Schur says. 
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Like other community colleges 
participating in the Every 
Learner grant which are part 
of the ATD Network, the seven 
institutions profiled in this 

case study have committed 
to engaging in bold, holistic, 
and sustainable institutional 
change across multiple 
institutional areas and priorities. 
Efforts to implement adaptive 
courseware support students 
in the gateway math courses 
in which poverty-impacted 
and racially minoritized 
students are often underrepresented at many of 
these institutions. They also reflect the importance of 
several key cornerstones of institutional change, 
including leveraging data and technology to 
support student success and equitable student 
outcomes and building a culture of excellence in 
teaching and learning. 


AID‘s Institutional Capacity Framework and Institutional 
Change Assessment Tool (ICAT) outlines seven essential 
institutional capacities required to create a student- 
focused culture that promotes student success, One 
focuses specifically on teaching and learning and the 
commitment to engaging full-time and adjunct faculty 
in examinations of pedagogy, meaningful professional 
development, and a central role for faculty as change 
agents within the institution. Building capacity in this 
area is crucial because, as ATD President Dr. Karen A. 
Stout recently asserted, “focusing on teaching and 
learning is still not central to the field’s overall theory of 
change. We still have much more to do to build a deep 
focus on pedagogy and to support our colleges in 


building a culture of teaching and learning excellence.” 


To foster this culture of teaching and learning ex- 
cellence, AID’s Teaching & Learning Toolkit: A Re- 
search-Based Guide to Building a Culture of Teaching 
& Learning Excellence is centered on four cornerstones 
of excellence that provide a forward-looking vision that 
campuses can use to inform their work. 


Initiatives such as Every Learner provide important 
resources and supports to community colleges and 
the time, space, support, and resources to explore 
innovative pedagogical approaches to improving 
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Pe BUILDING ON-AID Ss CORNERSTONES OF EXCELLENCE 


student learning and outcomes. 
They also offer sustained 
opportunities to build on these 
cornerstones of excellence. In this 
case, the grant provided faculty 
at participating institutions with 
the time and resources needed to 
evaluate, implement, and modify 
their use of adaptive courseware 
to best serve students. 


Efforts to provide needed 
corequisite support and basic 
skills building for students 
exemplifies the importance of 
institutional efforts to empower 
faculty to consider, adapt, test, and refine new 
approaches fo fit their campus context and the 
needs of their students. In this case, these efforts 
focused on ensuring more equitable outcomes for 
students struggling to succeed in gateway courses, 
particularly at those institutions which had eliminated 
developmental education courses as part of state- 
level or institutional reforms. “It was intended to take 
care of students on the back end so they could 
catch up and (be) all the same in the classroom,” 
says Nicholas Schur, math department chair at Miami 
Dade College’s (MDC) Kendall Campus. 


Faculty- led efforts to identify evidence-based instruc- 
tional practices that fostered student learning were 
centered in efforts to align courseware to curriculum 
and out-of-class supports such as peer tutoring, as 
well as ongoing attempts to adjust grading thresholds 
and other components to ensure that students didn’t 
become frustrated by a lack of progress within the 
courseware. “If was challenging to develop how we 
were going to use it (in ways) ... that wouldn’t overbur- 
den students but be enough,” says Christine McDon- 
ald, a master instructor in mathematics at Indian River 
State College (IRSC) in Florida. 


Faculty also worked to use data from adaptive 
courseware to monitor student progress and refocus 
supports and the content of their courses. “You need 
to use the data not to target students, but to coach or 
mentor them,” says Sarah Wyatt, associate professor of 
mathematics at Indian River State College. “You need 
the instructor there coaching the students through the 
course for it to be effective.” 


LESSONS LEARNED 


Keys to the successful implementation of adaptive 
courseware in math courses across Every Learner sites: 


e Consider how implementation fits 
in with other institutional initiatives. 
Since math is the focus of many 
initiatives, including but not limited 
to developmental education 
reforms, it’s vital fo understand 
where adaptive software fits in— 
and where it doesn’t. 


“Initiative fatigue was a constant 
point of conversation, especially 
with math,” says Nathan Smith, 

a faculty member and grant 
manager at Houston Community 
College in Texas, where faculty 
avoided using adaptive 
courseware in introductory-level 
courses To avoid conflicts with other 
reform initiatives. “To be successful, 
pilots need to be intentionally 
integrated into a department's 
priorities... if it isn’, it’s hard to 
maintain focus and enthusiasm.” 
Amarillo College was in a similar 
redesign of developmental math 
and subsequenily stepped back 
from using adaptive tools in those 
courses for that reason. 


Adaptive courseware isn’t a 
replacement for effective instruction. 
In math more than other subjects, 
even students who gave the 
technology high marks stressed the 
importance of the instructor; many 
said in interviews that the quality of 
the instructor was more important 
than the presence of adaptive 
tools. “| don’t think the courseware 
can do it by itself,” says Miami Dade 
College student Emily Gonzalez. 


e Faculty leadership is 
critical in ensuring effective 
implementation. Faculty must 
map their curriculum to ensure 
that courseware mirrors the 
desired units and objectives, 
advises Sarah Wyatt, associate 
professor of mathematics at 
Indian River State College (IRSC): 
“Go through the curriculum step 
by step... fo make sure it’s in 
line with what the department 
wants,” she says. 


EVERY LEARNER EVERYWHERE * ACHIEVING THE DREAM 9 


What Worked Well: 


Differentiation. Particularly on campuses which 
have eliminated developmental education, 
adaptive courseware helped students build 
prerequisite skills outside of class. “It targets what 
they need to practice,” says Christine McDonald, 
a master instructor in mathematics at Indian 
River State College (IRSC) in Florida. “It helps 
direct where they need to put their attention to 
be successful on coming tests.” Conversely, the 
people who “already know the material and 
have a decent math background get through 
the assignments quickly,” she adds. 


Ensuring students maintain an appropriate 
cadence. Requiring students to reach a 

set proficiency level before progressing To 

Tests or moving forward helped students 

keep up with course requirements. Kati 

Dobeck, a mathematics professor at Lorain 
County Community College (LCCC) in Ohio, 
acknowledges that students found the unit-level 
assignmenis stressful, but added that previously 
“students were really stressed when it came to 
exam time, because they hadn’t had the chance 
To synthesize things yet. That’s where | like to shift 


MMM ADAPTIVE COURSEWARE IN PRACTICE: SUCCESSES AND CHALLENGES 


the stressful experience, so when they get to the 
exam, they're like, ‘that’s not so bad’,” she says. 


Understanding student needs. Faculty who 
reviewed the data on student performance 
generated by adaptive products found that 

it helped them pinpoint support to individual 
students and shape in-class instruction to 
address the topics where the largest numbers 
of students struggled. “Out of the gate, we 
were having more meaningful conversations 
about what students need,” says Sarah Wyatt, 
associate professor of mathematics at IRSC. 


Cost savings. Faculty across campuses pointed 

To adaptive courseware’s low cost compared to 
textbooks, although some who had previously used 
non-adcaptive versions of the same courseware 
reported little or no change in costs, which were 
around $40 to $70 per student on many campuses. 


Accountability. Faculty said that the variability 
of questions delivered by adaptive courseware 
helped ensure that students did assigned work 
and were prepared for exams. “The bank has 
AO0 questions. It holds students accountable,” 
Dobeck says. 
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Ongoing Challenges: 


Addressing student frustration. Faculty frequently 
reported students being overwhelmed by the 
number of skills They had to master and how 
related assignments and tasks were presented 
by courseware. “It was structured in ways that 
students could throw their hands up in the air 
and say they can’t do the class,” says Nicholas 
Schur, math department chair at Miami Dade 
College’s (MDC) Kendall Campus. For example, 
some courseware highlights the full list of quizzes 
students will have to take throughout the 
semester after completing an initial diagnostic 
Test. “Some (quizzes) just have seven questions, 
but it does look overwhelming when you see this 
humongous list,” McDonald says. 


To address issues with students becoming 
frustrated by their lack of progress, Schur 
ultimately overruled some of the courseware’s 
efforts fo force constant retakes. The goal, he 
says, is to strike a balance between remediation 
while “at the same time pushing forward.” 


Even students who understood the value of 
retakes said that at times the courseware would 
slow progress for minor, sometimes mechanical, 
mistakes. “If you get a math question wrong, you 
should redo and redo it,” says Broward College 
student Valeska De Lang. “But if you accidentally 
put the wrong parentheses (and it counts it as 
wrong)? It’s great in the long run because it 
Teaches you fo look over things, but when you 
have a lot of assignments and have to do other 
things, in the moment it is frustrating.” 


Lack of higher-level materials. While adaptive 
courseware is effective at helping students 
reinforce and master core skills, faculty said it 
may not be ready for the more conceptual work 
required in higher-level courses like calculus. “| 
wish adaptive had more conceptual questions, 
or asked (questions like) what is the first step to 
solve the equation?” Wyatt says. 


Onboarding. Some students—even recent 
high school graduates accustomed to online 


classes—found working in adaptive courseware 
unfamiliar. “| wasn’t used to doing that kind 

of thing for homework,” says lvan Ondoa, a 
student at Houston Community College. Faculty 
stressed the need to explain the purpose of 
adaptive exercises to students. “Theyre used to 
(standardized) assignments,” Wyatt says. “We 
went on for a very long time explaining what they 
needed to do and the connection to the grade.” 


Challenges with planning. While traditional 
courseware often describes how long 
assignments should take students, adaptive 
assignments defy easy estimates. “With regular 
homework, | assigned 15 questions and can 
estimate how long if will take,” Wyatt says. “With 
adaptive, one student has 20 questions, another 
has 10. Some students are very low-skill, and it’s 
not going To take them a minute per question.” 
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Students also stress that alignment 
is critical in ensuring that 
courseware is effective. “It works 
really well as long as what the 
professor is teaching and what's in 
the software is properly aligned,” 
says Gonzalez. “Otherwise, you 
don’t know what way you should 
be solving problems.” 


Amarillo College instructor of 
mathematics, science, and 
engineering Gale Brewer saw 
this firsthand after making mid- 
course adjustments. “Students did 
(adaptive work) but didn’t see it 
as beneficial until | tweaked it so 
it matched what | was doing in 
college algebra,” she says. And 
Christine McDonald, a master 
instructor in mathematics at IRSC, 
says that the Florida college had 
better results in courses where 
students had experience with 
college-level math than the 
introductory course, despite 
adaptive courseware’s promise 
to support students with lower 
skill levels. “It could be we didn’t 
find the right way to use it in (that 
course),” she says. 
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e Prepare students to use adaptive 
software. Wyatt began her in- 
person classes in the computer 
lab to ensure students log into the 
courseware and understand how to 
use it. “The software is designed to 
help them, but what gets in the way 
is that they get so overwhelmed 
from the get-go. You have to 
coach them to stick with and get 
through the adaptive (elements),” 
she says. In similar fashion, HCCS 
developed a student onboarding 
module which was added to its 
learning management system to 
guide students and faculty. 


Monitoring student progress is 
critical. “You need to use the 

data not to target students, but 

to coach or mentor them,” Wyatt 
says. “You need the instructor there 
coaching the students through the 
course for it to be effective.” Kati 
Dobeck, a mathematics professor 
at Lorain County Community 
College (LCCC), for example, 
monitors students’ most-missed 
questions in the courseware to 
plan the focus of her synchronous 
sessions in an online math course. 


Along with identifying which 
students are struggling, McDonald 
says it’s also important to monitor 
how students are using the 
courseware. “Pointing to the 
example of a student who kept 
unsuccessfully retaking a quiz 
instead of going back to do 
practice problems, she says, “It’s our 
responsibility to show them what 
they need to do to be successful... 
instead of doing the inefficient 
thing over and over again.” 


Dobeck agrees, saying that she 
reaches out to students who are 
spending large amounts of time 
on a specific assignment to guide 
them to other options. “It’s not my 
intention for it to take five hours— 
maybe go talk to a tutor and it 
would take less time,” she says. 


Using the courseware in class. 
Despite its primary use case as 
guided homework and practice, 
faculty and students stressed 
the value of having some class 
time dedicated to working in the 
courseware. Wyatt held classes 


in the computer lab regularly 

to watch students work. “It was 
helpful for me to see who was 
struggling and making progress in 
the courseware,” she says. 


Ensure that faculty use the 
courseware themselves. Many 
students stressed the importance 
of having instructors try the 
software themselves so they 
understand the range of questions 
included in adaptive exercises and 
create alignment between those 
exercises and their other class 
activities. Dobeck agrees. Pointing 
to her own attempt at taking the 
start-of-course diagnostic test, she 
says, “| got an 87 percent, and | 
know the stuff.” More to the point, 
it made her realize that the default 
passing threshold of 80 percent 
wasn’t appropriate, resulting in 

her changing grading standards 
for the course in ways that more 
students progressed through the 
courseware in a timely fashion. 
“You'd be surprised at how tricky 
the questions were,” Dobeck says. 


G61 don’t know if professors see everything we have 
to do (in the courseware). Last week, we had fo turn 
in two assignments—one with eight questions and 
another with 17. The same week we did a practice 


test, which helps us for the test, but if was 35 questions. 

Everyone said there were (types of) questions we 

didn’t see in class that were in the pretest.99 
—Geolmary Suazo, Broward College student 
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CONCLUSION 


Faculty at participating institutions said that the Every 
Learner grant support provided them with the time 
and resources needed to evaluate and implement 


adaptive courseware. 


While the coronavirus pandemic 
limited their ability to fully measure 
its impact, anecdotal evidence 
suggests the technology was 
beneficial. Preliminary data 
suggests that adaptive courseware 
helped bring students up to 
college-level math in introductory 
classes at MDC, says Schur. At 
Indian River State College, one 
math instructor saw pass rates 
jump from 40 percent to 60 percent 
after making adaptive study a 
prerequisite for taking graded 
quizzes. Faculty members across 
participating sites reported that 
overall grades improved, while 
some said that more students 
completed homework, with the 
technology helping focus their 
attention on the areas they 
needed most. 


“The beauty of adaptive is 

that it’s laser focused on what 
you need to know for the 
semester to be successful—it 
eliminates busywork,” says Sarah 
Wyatt, associate professor of 
mathematics at IRSC. “| wouldn’t 
say engagement is more or less, 
but it makes the focus better.” 


Experiences scaling adaptive 
courseware varied from campus 
to campus. Some institutions 
scaled back the use of adaptive 
courseware in developmental 
math courses because of conflicts 
with ongoing redesign efforts. 

At others, math faculty made 
deliberate efforts to ensure that 
college algebra sections taught by 
adjunct faculty were aligned with 
adaptive courseware by creating 
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master course shells and/or daily 
lecture notes to ensure consistency 
across sections taught by full-time 
and adjunct faculty. “The idea was 
that the platform would make their 
lives easier,” MDC’s Schur says. 
“We wanted a turnkey approach.” 
At other colleges, implementation 
was less consistent, depending 

on individual adjunct faculty 
members’ willingness to 
experiment with new technology 


or definitions of academic 
freedom. 


Now a second-year student at 
Broward, Suazo credits the efforts 
her professor made to introduce 
her class to the courseware to her 
success in the initial math class she 
took. “She explained it the first time 
and explained it well,” she says. 
“She gave me the vibe that I’d 
understand everything.” 
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